Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adding return state option to recurrent layers #2557

Open
wants to merge 3 commits into
base: master
Choose a base branch
from

Conversation

MartinuzziFrancesco
Copy link
Contributor

@MartinuzziFrancesco MartinuzziFrancesco commented Dec 24, 2024

Addressing the return_state as discussed in #2514.

PR Checklist

  • Tests are added
  • Entry in NEWS.md
  • Documentation, if applicable

@MartinuzziFrancesco MartinuzziFrancesco marked this pull request as ready for review December 25, 2024 17:50
@CarloLucibello
Copy link
Member

tests are failing

@MartinuzziFrancesco
Copy link
Contributor Author

MartinuzziFrancesco commented Dec 26, 2024

is this something that has to be addressed at the Functor level? the fail is at https://github.com/FluxML/Flux.jl/blob/master/test/test_utils.jl#L66 with

MethodError: no method matching RNN(::RNNCell{typeof(tanh), Matrix{Float64}, Matrix{Float64}, Vector{Float64}})

so I guess it's trying to rebuild the struct but it can't since it doesn't have the return_state parametrization information. I can add a workaround like

RNN(rnn::RNNCell) = RNN{false, RNNCell}(rnn)

but this wouldn't solve it at the level that it's needed. The tests would still fail for return_state = true

@CarloLucibello
Copy link
Member

yes this needs a custom functor implementation. Maybe something like

Functors.functor(rnn::RNN{S}) = (; rnn.cell), ch -> RNN{S}(ch.cell)

(not tested)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants